Fusion recognition of palmprint and palm vein based on modal correlation.

Biometric authentication prevents losses from identity misuse in the artificial intelligence (AI) era. The fusion method integrates palmprint and palm vein features, leveraging their stability and security and enhances counterfeiting prevention and overall system efficiency through multimodal correlations. However, most of the existing multi-modal palmprint and palm vein feature extraction methods extract only feature information independently from different modalities, ignoring the importance of the correlation between different modal samples in the class to the improvement of recognition performance. In this study, we addressed the aforementioned issues by proposing a feature-level joint learning fusion approach for palmprint and palm vein recognition based on modal correlations. The method employs a sparse unsupervised projection algorithm with a "purification matrix" constraint to enhance consistency in intra-modal features. This minimizes data reconstruction errors, eliminating noise and extracting compact, and discriminative representations. Subsequently, the partial least squares algorithm extracts high grayscale variance and category correlation subspaces from each modality. A weighted sum is then utilized to dynamically optimize the contribution of each modality for effective classification recognition. Experimental evaluations conducted for five multimodal databases, composed of six unimodal databases including the Chinese Academy of Sciences multispectral palmprint and palm vein databases, yielded equal error rates (EER) of 0.0173%, 0.0192%, 0.0059%, 0.0010%, and 0.0008%. Compared to some classical methods for palmprint and palm vein fusion recognition, the algorithm significantly improves recognition performance. The algorithm is suitable for identity recognition in scenarios with high security requirements and holds practical value.

Enhancing Face Recognition With Detachable Self-Supervised Bypass Networks.

Attributed to the development of deep networks and abundant data, automatic face recognition (FR) has quickly reached human-level capacity in the past few years. However, the FR problem is not perfectly solved in case of large poses and uncontrolled occlusions. In this paper, we propose a novel bypass enhanced representation learning (BERL) method to improve face recognition under unconstrained scenarios. The proposed method integrates self-supervised learning and supervised learning together by attaching two auxiliary bypasses, a 3D reconstruction bypass and a blind inpainting bypass, to assist robust feature learning for face recognition. Among them, the 3D reconstruction bypass enforces the face recognition network to encode pose independent 3D facial information, which enhances the robustness to various poses. The blind inpainting bypass enforces the face recognition network to capture more facial context information for face inpainting, which enhances the robustness to occlusions. The whole framework is trained in end-to-end manner with two self-supervised tasks above and the classic supervised face identification task. During inference, the two auxiliary bypasses can be detached from the face recognition network, avoiding any additional computational overhead. Extensive experimental results on various face recognition benchmarks show that, without any cost of extra annotations and computations, our method outperforms state-of-the-art methods. Moreover, the learnt representations can also well generalize to other face-related downstream tasks such as the facial attribute recognition with limited labeled data.

Biometric identification of dairy cows via real-time facial recognition.

Biometrics methods, which currently identify humans, can potentially identify dairy cows. Given that animal movements cannot be easily controlled, identification accuracy and system robustness are challenging when deploying an animal biometrics recognition system on a real farm. Our proposed method performs multiple-cow face detection and face classification from videos by adjusting recent state-of-the-art deep-learning methods. As part of this study, a system was designed and installed at four meters above a feeding zone at the Volcani Institute's dairy farm. Two datasets were acquired and annotated, one for facial detection and the second for facial classification of 77 cows. We achieved for facial detection a mean average precision (at Intersection over Union of 0.5) of 97.8% using the YOLOv5 algorithm, and facial classification accuracy of 96.3% using a Vision-Transformer model with a unique loss-function borrowed from human facial recognition. Our combined system can process video frames with 10 cows' faces, localize their faces, and correctly classify their identities in less than 20 ms per frame. Thus, up to 50 frames per second video files can be processed with our system in real-time at a dairy farm. Our method efficiently performs real-time facial detection and recognition on multiple cow faces using deep neural networks, achieving a high precision in real-time operation. These qualities can make the proposed system a valuable tool for an automatic biometric cow recognition on farms.

Enhanced multimodal biometric recognition systems based on deep learning and traditional methods in smart environments.

In the field of data security, biometric security is a significant emerging concern. The multimodal biometrics system with enhanced accuracy and detection rate for smart environments is still a significant challenge. The fusion of an electrocardiogram (ECG) signal with a fingerprint is an effective multimodal recognition system. In this work, unimodal and multimodal biometric systems using Convolutional Neural Network (CNN) are conducted and compared with traditional methods using different levels of fusion of fingerprint and ECG signal. This study is concerned with the evaluation of the effectiveness of proposed parallel and sequential multimodal biometric systems with various feature extraction and classification methods. Additionally, the performance of unimodal biometrics of ECG and fingerprint utilizing deep learning and traditional classification technique is examined. The suggested biometric systems were evaluated utilizing ECG (MIT-BIH) and fingerprint (FVC2004) databases. Additional tests are conducted to examine the suggested models with:1) virtual dataset without augmentation (ODB) and 2) virtual dataset with augmentation (VDB). The findings show that the optimum performance of the parallel multimodal achieved 0.96 Area Under the ROC Curve (AUC) and sequential multimodal achieved 0.99 AUC, in comparison to unimodal biometrics which achieved 0.87 and 0.99 AUCs, for the fingerprint and ECG biometrics, respectively. The overall performance of the proposed multimodal biometrics outperformed unimodal biometrics using CNN. Moreover, the performance of the suggested CNN model for ECG signal and sequential multimodal system based on neural network outperformed other systems. Lastly, the performance of the proposed systems is compared with previously existing works.

Finger Vein Identification Based on Large Kernel Convolution and Attention Mechanism.

FV (finger vein) identification is a biometric identification technology that extracts the features of FV images for identity authentication. To address the limitations of CNN-based FV identification, particularly the challenge of small receptive fields and difficulty in capturing long-range dependencies, an FV identification method named Let-Net (large kernel and attention mechanism network) was introduced, which combines local and global information. Firstly, Let-Net employs large kernels to capture a broader spectrum of spatial contextual information, utilizing deep convolution in conjunction with residual connections to curtail the volume of model parameters. Subsequently, an integrated attention mechanism is applied to augment information flow within the channel and spatial dimensions, effectively modeling global information for the extraction of crucial FV features. The experimental results on nine public datasets show that Let-Net has excellent identification performance, and the EER and accuracy rate on the FV_USM dataset can reach 0.04% and 99.77%. The parameter number and FLOPs of Let-Net are only 0.89M and 0.25G, which means that the time cost of training and reasoning of the model is low, and it is easier to deploy and integrate into various applications.

Enhancement, integration, expansion: Activating representation of detailed features for occluded person re-identification.

A proposed method, Enhancement, integration, and Expansion, aims to activate the representation of detailed features for occluded person re-identification. Region and context are two important and complementary features, and integrating them in an occluded environment can effectively improve the robustness of the model. Firstly, a self-enhancement module is designed. Based on the constructed multi-stream architecture, rich and meaningful feature interference is introduced in the feature extraction stage to enhance the model's ability to perceive noise. Next, a collaborative integration module similar to cascading cross-attention is proposed. By studying the intrinsic interaction patterns of regional and contextual features, it adaptively fuses features across streams and enhances the diverse and complete representation of internal information. The module is not only robust to complex occlusions, but also mitigates the feature interference problem due to similar appearances or scenes. Finally, a matching expansion module that enhances feature discriminability and completeness is proposed. Providing more stable and accurate features for recognition. Compared with state-of-the-art methods on two occluded and holistic datasets, the proposed method is proved to be advanced and the effectiveness of the module is proved by extensive ablation studies.

A comparative analysis of multi-biometrics performance in human and action recognition using silhouette thermal-face and skeletal data.

Biometrics is a field that has been given importance in recent years and has been extensively studied. Biometrics can use physical and behavioural differences that are unique to individuals to recognize and identify them. Today, biometric information is used in many areas such as computer vision systems, entrance systems, security and recognition. In this study, a new biometrics database containing silhouette, thermal face and skeletal data based on the distance between the joints was created to be used in behavioural and physical biometrics studies. The fact that many cameras were used in previous studies increases both the processing intensity and the material cost. This study aimed to both increase the recognition performance and reduce material costs by adding thermal face data in addition to soft and behavioural biometrics with the optimum camera. The presented data set was created in accordance with both motion recognition and person identification. Various data loss scenarios and multi-biometrics approaches based on data fusion have been tried on the created data sets and the results have been given comparatively. In addition, the correlation coefficient of the motion frames method to obtain energy images from silhouette data was tested on this dataset and yielded high-accuracy results for both motion and person recognition.

Metabolite monitoring concept for the biometric identification of individuals from the skin surface.

This study aims at proof of concept that constant monitoring of the concentrations of metabolites in three individuals' sweat over time can differentiate one from another at any given time, providing investigators and analysts with increased ability and means to individualize this bountiful biological sample. A technique was developed to collect and extract authentic sweat samples from three female volunteers for the analysis of lactate, urea, and L-alanine levels. These samples were collected 21 times over a 40-day period and quantified using a series of bioaffinity-based enzymatic assays with UV-vis spectrophotometric detection. Sweat samples were simultaneously dried, derivatized, and analyzed by a GC-MS technique for comparison. Both UV-vis and GC-MS analysis methods provided a statistically significant MANOVA result, demonstrating that the sum of the three metabolites could differentiate each individual at any given day of the time interval. Expanding upon previous studies, this experiment aims to establish a method of metabolite monitoring as opposed to single-point analyses for application to biometric identification from the skin surface.

A Large-Scale Clinical Benchmark of ResNet-based Deep Models for Newborn Face Recognition.

Newborn face recognition is a meaningful application for obstetrics in the hospital, as it enhances security measures against infant swapping and abduction through authentication protocols. Due to limited newborn face datasets, this topic was not thoroughly studied. We conducted a clinical trial to create a dataset that collects face images from 200 newborns within an hour after birth, namely NEWBORN200. To our best knowledge, this is the largest newborn face dataset collected in the hospital for this application. The dataset was used to evaluate the four latest ResNet-based deep models for newborn face recognition, including ArcFace, CurricularFace, MagFace, and AdaFace. The experimental results show that AdaFace has the best performance, obtaining 55.24% verification accuracy at 0.1% false accept rate in the open set while achieving 78.76% rank-1 identification accuracy in a closed set. It demonstrates the feasibility of using deep learning for newborn face recognition, also indicating the direction of improvement could be the robustness to varying postures.

Comparison of Wrist and Forearm EMG for Multi-day Biometric Authentication.

Recently, electromyography (EMG) has been established as a promising new biometric trait that provides a unique dual mode security: biometrics and knowledge. For authentication that is used daily and long-term by general consumers, the wrist is a suitable location, which could be easily integrated into the existing form of smartwatches and fitness trackers. However, current EMG-based biometrics still follow the historical path of powered prosthetics research, where EMG signals were usually recorded from forearm positions. Moreover, the robustness of EMG processing algorithms across multiple days is still an open problem that needs to be addressed before for long-term reliable use. This study intends to investigate the difference in authentication performance between wrist and forearm EMG signals, in a within-day and two cross-day analyses. Our open dataset (GRABMyo dataset) was used to examine this difference, which contains forearm and wrist EMG data collected from 43 participants over three different days with long separation (Days 1, 8, and 29). The results showed wrist EMG signals led to at least comparable with forearm EMG signals in within-day Equal-error rate (EER). In cross-day analysis, the EER of the wrist EMG signals was higher than that of forearm signals. In general, the low median EER (<0.1) of wrist EMG in cumulative cross-day analysis demonstrates the promise of using wrist EMG signals for authentication in long-term applications.

A Scalable ECG Identification System Based on Locality-Sensitive Hashing.

Electrocardiograms (ECGs) have the inherent property of being intrinsic and dynamic and are shown to be unique among individuals, making them promising as a biometric trait. Although many ECG biometric recognition approaches have demonstrated accurate recognition results in small enrollment sets, they can suffer from performance degradation when many subjects are enrolled. This study proposes an ECG biometric identification system based on locality-sensitive hashing (LSH) that can accommodate a large number of registrants while maintaining satisfactory identification accuracy. By incorporating the concept of LSH, the identity of an unknown subject can be recognized without performing vector comparisons for all registered subjects. Moreover, a kernel density estimator-based method is used to exclude unregistered subjects. The ECGs of 285 subjects from the PTB dataset were used to evaluate the proposed scheme's performance. Experimental results demonstrated an IR and EER of 99% and 4%, respectively, when Nen/Nid = 15/3.

Comparison of the ocular ultrasonic and optical biometry devices in the different quality measurements

Purpose: To compare the reliability and agreement of axial length (AL), anterior chamber depth (ACD), and lens thickness (LT) measurements obtained with optical biometry based on swept-source optical coherence tomography (IOLMaster 700; Carl Zeiss, Germany) and an ultrasound biometry device (Nidek; US-4000 Echoscan, Japan) in different qualities of AL measurement. Methods: A total of 239 consecutive eyes of 239 cataract surgery candidates with a mean age of 56 ± 14 years were included. The quality measurements were grouped according to the quartiles of SD of the measured AL by IOLMaster 700. The first and fourth quartile's SD are defined as high and low-quality measurement, respectively, and the second and third quartiles’ SD is defined as moderate-quality. Results: The reliability of AL and ACD between the two devices in all patients and in different quality measurement groups was excellent with highly statistically significant (AL: all ICC=0.999 and P<0.001, ACD: all ICC>0.920 and P<0.001). AL and ACD in all quality measurements showed a very strong correlation between devices with highly statistically significant. However, there was poor (ICC=0.305), moderate (ICC=0.742), and good (ICC=0.843) reliability in measuring LT in low-, moderate-, and high-quality measurements, respectively. LT showed a very strong correlation (r = 0.854) with highly statistically significant (P<0.001) between devices only in patients with high-quality measurements. Conclusions: AL and ACD of the IOLMaster700 had outstanding agreements with the US-4000 ultrasound in different quality measurements of AL and can be used interchangeably. But LT should be used interchangeably cautiously only in the high-quality measurements group. (AU)

An Improved Multimodal Biometric Identification System Employing Score-Level Fuzzification of Finger Texture and Finger Vein Biometrics.

This research work focuses on a Near-Infra-Red (NIR) finger-images-based multimodal biometric system based on Finger Texture and Finger Vein biometrics. The individual results of the biometric characteristics are fused using a fuzzy system, and the final identification result is achieved. Experiments are performed for three different databases, i.e., the Near-Infra-Red Hand Images (NIRHI), Hong Kong Polytechnic University (HKPU) and University of Twente Finger Vein Pattern (UTFVP) databases. First, the Finger Texture biometric employs an efficient texture feature extracting algorithm, i.e., Linear Binary Pattern. Then, the classification is performed using Support Vector Machine, a proven machine learning classification algorithm. Second, the transfer learning of pre-trained convolutional neural networks (CNNs) is performed for the Finger Vein biometric, employing two approaches. The three selected CNNs are AlexNet, VGG16 and VGG19. In Approach 1, before feeding the images for the training of the CNN, the necessary preprocessing of NIR images is performed. In Approach 2, before the pre-processing step, image intensity optimization is also employed to regularize the image intensity. NIRHI outperforms HKPU and UTFVP for both of the modalities of focus, in a unimodal setup as well as in a multimodal one. The proposed multimodal biometric system demonstrates a better overall identification accuracy of 99.62% in comparison with 99.51% and 99.50% reported in the recent state-of-the-art systems.

Audio-Visual Fusion Based on Interactive Attention for Person Verification.

With the rapid development of multimedia technology, personnel verification systems have become increasingly important in the security field and identity verification. However, unimodal verification systems have performance bottlenecks in complex scenarios, thus triggering the need for multimodal feature fusion methods. The main problem with audio-visual multimodal feature fusion is how to effectively integrate information from different modalities to improve the accuracy and robustness of the system for individual identity. In this paper, we focus on how to improve multimodal person verification systems and how to combine audio and visual features. In this study, we use pretrained models to extract the embeddings from each modality and then perform fusion model experiments based on these embeddings. The baseline approach in this paper involves taking the fusion feature and passing it through a fully connected (FC) layer. Building upon this baseline, we propose three fusion models based on attentional mechanisms: attention, gated, and inter-attention. These fusion models are trained on the VoxCeleb1 development set and tested on the evaluation sets of the VoxCeleb1, NIST SRE19, and CNC-AV datasets. On the VoxCeleb1 dataset, the best system performance achieved in this study was an equal error rate (EER) of 0.23% and a detection cost function (minDCF) of 0.011. On the evaluation set of NIST SRE19, the EER was 2.60% and the minDCF was 0.283. On the evaluation set of the CNC-AV set, the EER was 11.30% and the minDCF was 0.443. These experimental results strongly demonstrate that the proposed fusion method can significantly improve the performance of multimodal character verification systems.

Electrocardiogram-Based Biometric Identification Using Mixed Feature Extraction and Sparse Representation.

(1) Background: The ability to recognize identities is an essential component of security. Electrocardiogram (ECG) signals have gained popularity for identity recognition because of their universal, unique, stable, and measurable characteristics. To ensure accurate identification of ECG signals, this paper proposes an approach which involves mixed feature sampling, sparse representation, and recognition. (2) Methods: This paper introduces a new method of identifying individuals through their ECG signals. This technique combines the extraction of fixed ECG features and specific frequency features to improve accuracy in ECG identity recognition. This approach uses the wavelet transform to extract frequency bands which contain personal information features from the ECG signals. These bands are reconstructed, and the single R-peak localization determines the ECG window. The signals are segmented and standardized based on the located windows. A sparse dictionary is created using the standardized ECG signals, and the KSVD (K-Orthogonal Matching Pursuit) algorithm is employed to project ECG target signals into a sparse vector-matrix representation. To extract the final representation of the target signals for identification, the sparse coefficient vectors in the signals are maximally pooled. For recognition, the co-dimensional bundle search method is used in this paper. (3) Results: This paper utilizes the publicly available European ST-T database for our study. Specifically, this paper selects ECG signals from 20, 50 and 70 subjects, each with 30 testing segments. The method proposed in this paper achieved recognition rates of 99.14%, 99.09%, and 99.05%, respectively. (4) Conclusion: The experiments indicate that the method proposed in this paper can accurately capture, represent and identify ECG signals.

Human Micro-Expressions in Multimodal Social Behavioral Biometrics.

The advent of Social Behavioral Biometrics (SBB) in the realm of person identification has underscored the importance of understanding unique patterns of social interactions and communication. This paper introduces a novel multimodal SBB system that integrates human micro-expressions from text, an emerging biometric trait, with other established SBB traits in order to enhance online user identification performance. Including human micro-expression, the proposed method extracts five other original SBB traits for a comprehensive representation of the social behavioral characteristics of an individual. Upon finding the independent person identification score by every SBB trait, a rank-level fusion that leverages the weighted Borda count is employed to fuse the scores from all the traits, obtaining the final identification score. The proposed method is evaluated on a benchmark dataset of 250 Twitter users, and the results indicate that the incorporation of human micro-expression with existing SBB traits can substantially boost the overall online user identification performance, with an accuracy of 73.87% and a recall score of 74%. Furthermore, the proposed method outperforms the state-of-the-art SBB systems.

Populational Variations of Cheiloscopy Patterns: A cross-sectional observation pilot study.

Purpose Lip prints are unique and have potential for use as a human identifier. The purpose of this study was to observe possible cheiloscopy differences of individuals with and without parafunctional oral habits such as smoking, vaping, playing a wind instrument or using an asthma inhaler.Methods This IRB approved blinded cross-sectional observation pilot study collected lip prints from sixty-six individuals, three of which were excluded. Participants cleansed their lips, then lipstick was applied to the vermillion zones of the upper and lower lips. Adhesive tape was applied to the lips and prints were transferred to white bond paper for viewing purposes. Each set of included lip prints was divided into quadrants and dichotomized into a group of those with an oral parafunctional habit or with no such habits. Each quadrant sample was then manually analyzed and classed according to the gold standard Suzuki and Tsuchihashi system.Results A total of 252 dichotomized lip print quadrants (with habits n=76, 30.2%, and without habits n=176, 69.8%) were analyzed. Type II patterns were the most common for examined quadrant samples; however, no statistically significant differences (Pearson's chi-squared test, p=0.366) were observed between pattern classifications of samples with and without parafunctional oral habits.Conclusion There is no statistically significant difference of lip print patterns between individuals with and without parafunctional oral habits. Further research on populational variations is needed for cheiloscopy to aid in human identifications.

A Symmetrical Siamese Network Framework With Contrastive Learning for Pose-Robust Face Recognition.

Face recognition has achieved remarkable success owing to the development of deep learning. However, most of existing face recognition models perform poorly against pose variations. We argue that, it is primarily caused by pose-based long-tailed data - imbalanced distribution of training samples between profile faces and near-frontal faces. Additionally, self-occlusion and nonlinear warping of facial textures caused by large pose variations also increase the difficulty in learning discriminative features of profile faces. In this study, we propose a novel framework called Symmetrical Siamese Network (SSN), which can simultaneously overcome the limitation of pose-based long-tailed data and pose-invariant features learning. Specifically, two sub-modules are proposed in the SSN, i.e., Feature-Consistence Learning sub-Net (FCLN) and Identity-Consistence Learning sub-Net (ICLN). For FCLN, the inputs are all face images on training dataset. Inspired by the contrastive learning, we simulate pose variations of faces and constrain the model to focus on the consistent areas between the original face image and its corresponding virtual pose face images. For ICLN, only profile images are used as inputs, and we propose to adopt Identity Consistence Loss to minimize the intra-class feature variation across different poses. The collaborative learning of two sub-modules guarantees that the parameters of network are updated in a relatively equal probability between near-frontal face images and profile images, so that the pose-based long-tailed problem can be effectively addressed. The proposed SSN shows comparable results over the state-of-the-art methods on several public datasets. In this study, LightCNN is selected as the backbone of SSN, and existing popular networks also can be used into our framework for pose-robust face recognition.

Face Off: Overcoming the Fifth Amendment Conflict Between Cybersecurity and Self-Incrimination.

The Founders included the privilege against self-incrimination in the Constitution to protect individual privacy and ensure a fair judicial process. Courts have failed U.S. citizens by neglecting to protect them from compelled unlocking of biometrically encrypted devices. This inaction has created a loophole that contradicts the framework of the privilege against self-incrimination. To correct this mistake courts should reconsider the trend they have set for the Constitution and the Fifth Amendment and consider adopting a forward-thinking cybersecurity lens to conclude that biometric authentication is testimonial. Courts should consider that biometric encryption is akin to a compelled password entry for the purposes of the foregone conclusion doctrine. The foregone conclusion doctrine should be applied in limited circumstances with a specific and high burden of proof so that the "jealous protection of the privilege against self-incriminating testimony" can be preserved. Allowing law enforcement such easy access to smart devices narrows Fifth Amendment protections and the expansive foregone conclusion exception is contrary to both principles of cybersecurity and the spirit of the Fifth Amendment. Courts should move to remediate this at once. These liberties and values can only be guaranteed by courts that are willing to take on cases with issues revolving around biometric encryption, the Fifth Amendment, and the foregone conclusion doctrine.

Photoplethysmogram Biometric Authentication Using a 1D Siamese Network.

In the head-mounted display environment for experiencing metaverse or virtual reality, conventional input devices cannot be used, so a new type of nonintrusive and continuous biometric authentication technology is required. Since the wrist wearable device is equipped with a photoplethysmogram sensor, it is very suitable for use for nonintrusive and continuous biometric authentication purposes. In this study, we propose a one-dimensional Siamese network biometric identification model using a photoplethysmogram. To maintain the unique characteristics of each person and reduce noise in preprocessing, we adopted a multicycle averaging method without using a bandpass or low-pass filter. In addition, to verify the effectiveness of the multicycle averaging method, the number of cycles was changed and the results were compared. Genuine and impostor data were used to verify the biometric identification. We used the one-dimensional Siamese network to verify the similarity between the classes and found that the method with five overlapping cycles was the most effective. Tests were conducted on the overlapping data of five single-cycle signals and excellent identification results were observed, with an AUC score of 0.988 and an accuracy of 0.9723. Thus, the proposed biometric identification model is time-efficient and shows excellent security performance, even in devices with limited computational capabilities, such as wearable devices. Consequently, our proposed method has the following advantages compared with previous works. First, the effect of noise reduction and information preservation through multicycle averaging was experimentally verified by varying the number of photoplethysmogram cycles. Second, by analyzing authentication performance through genuine and impostor matching analysis based on a one-dimensional Siamese network, the accuracy that is not affected by the number of enrolled subjects was derived.